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(54) Moving object detection apparatus and method 



(57) This invention provides an image processing 
apparatus/method characterized by inputting image da- 
ta, detecting an object in the input image data, measur- 
ing the distance from the detected object to a predeter- 
mined position, and detecting a predetermined object 
on the basis of the measurement result. 



This invention also provides an image processing 
apparatus/method characterized by inputting image da- 
ta by image pickup means having an optical system, de- 
tecting an object in the input image data, controlling the 
optical system of the image pickup means, and detect- 
ing a predetermined object on the basis of the object 
detection result and the optical system control result. 
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Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates to an image 
processing apparatus and an image processing method 
of detecting a desired object from input image data. 

Related Background Art 

[0002] One conventional moving object detection ap- 
paratus detects an intruder or abnormality by detecting 
a moving object in an image being taken by a video cam- 
era for a monitoring purpose orthe like. In many instanc- 
es, the size of a moving object to be detected by such 
an apparatus is previously known. Therefore, it is desir- 
able to so design an apparatus that the apparatus de- 
tects only a moving object of a specific size. 
[0003] Unfortunately, in an image picked up by an im- 
age pickup device such as a video camera, the size of 
an object changes in accordance with the distance to 
the object or the magnification. 
[0004] This will be described below with reference to 
Fig. 1 . Referring to Fig. 1 , an image pickup system has 
an image pickup center 120 and an optical axis 127. 
Planes 121, 122, and 123 are perpendicular to the op- 
tical axis 1 27 and at distances of 1 , 2, and 3 m, respec- 
tively, from the image pickup center 1 20. Spheres 1 24, 
125, and 126 have a radius of 1/3 m, and their centers 
are in the planes 121, 122, and 123, respectively. The 
horizontal and vertical field angles of this image pickup 
system are 36.0° and 27.0°, respectively. Fig. 1 shows 
the horizontal field angle viewed from immediately 
above. Lines 1 29 and 1 30 indicate the field angle range 
viewed from the image pickup center 120. The angle 
formed by 1 30 - 1 20 - 1 29 is 36.0°. Both of 1 27 - 1 20 - 
129 and 127- 120 -130 form an angle of 18°. An image 
picked up by this image pickup system is formed by 640 
pixels (horizontal direction) x 480 lines (vertical direc- 

[0005] Fig. 2 shows the sizes of the spheres 1 24, 1 25, 
and 1 26 in an image frame captured by 640 x 480 pixels 
described above. As shown in Figs. 1 and 2, images of 
objects having exactly the same size have different siz- 
es in the frame in accordance with their distances from 
the image pickup center. Referring to Figs. 1 and 2, the 
sphere 124 shown in Fig. 1 occupies a horizontal field 
angle of about 19° (the angle formed by A - 120 - A') 
across its diameter and has an image size of about 327 
pixels in the frame. Similarly, the sphere 125 occupies 
about 9.5° (the angle formed by B - 120 - B 1 ) and has 
an image size of about 163 pixels. The sphere 126 oc- 
cupies about 6.4° and has an image size of 109 pixels. 
[0006] Fig. 3 shows results when the image pickup 
system optically changes its magnification. D-f - 120 - 
D,' indicates a field angle of about 36.0° obtained at a 



reference magnification of 1x. D 2 - 120 • 
a field angle of about 18.5° obtained when the magnifi- 
cation is 2X. D 3 - 1 20 - D 3 ' indicates a field angle of 
about 12.4° obtained when the magnification is 3X. 

s When the magnification is changed in this manner, a 
field angle corresponding to 640 pixels of the frame size 
of an image changes. When the magnification is in- 
creased, the image size of an object increases in pro- 
portion to the magnification. That is, the object size rei- 

10 ative to the image frame size increases. 

[0007] Accordingly, detection of an object of a specific 
size must be performed in consideration of the above 
phenomenon. 

[0008] Additionally, a monitoring area for a moving ob- 
15 ject to be detected by a moving object detection appa- 
ratus is often limited. So, it is desirable to allow the ap- 
paratus to detect a moving object only in a part of an 
image area being picked up. 

[0009] For example, the following moving object de- 

2f tection is possible. 

[0010] Fig. 4 shows an image taken at a certain fixed 
field angle by a video camera. Fig. 5 shows a detection 
area 1 01 set in the image shown in Fig. 4. This detection 
area 101 is composed of a plurality of rectangular areas 

2S 100 as a minimum unit including n x m pixels (e.g., 16 
x 1 2 pixels or 24 x 24 pixels). This detection area 1 01 is 
used to, e.g., detect an object which is intruding into an 
area surrounded by a fence 102 in the image shown in 
Fig. 4. 

so [0011] In the above prior art, however, the specific de- 
tection area 1 01 is set in an image picked up at a certain 
fixed angle, and image changes in this specific area are 
detected. Therefore, not only changes in a monitoring 
area to be detected but also changes which need not 

ss be detected or should not be detected are detected. 
[0012] For example, if an intruder 103 approaches the 
fence 102 as shown in Fig. 6, changes to be detected 
can be detected in the detection area 101. However, 
even if a moving object 1 04 exists far away (closer to 

40 an image pickup camera) from the detection point as 
shown in Fig. 7, changes in the detection area 101 are 
detected. That is, changes which should not be detected 
are detected. 

[0013] To avoid this situation, it is possible to improve 
45 the setting of the field angle, e.g., install a video camera 
above the monitoring area. Generally, however, the set- 
ting of the field angle is not always selectable. Also, ac- 
cidental detection of an object flying overthe monitoring 
area is unavoidable. 

SUMMARY OF THE INVENTION 

[0014] The present invention has been made in con- 
sideration of the above situation and has as a concern 
ss to provide an image processing apparatus/method ca- 
pable of detecting an object (e.g., an object of a prede- 
termined size or an object within a predetermined dis- 
tance range from a predetermined object) desired by a 
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user from input image data. 

[001 5] According to one preferred embodiment of the 
present invention, an image processing apparatus/ 
method is characterized by inputting image data, detect- 
ing an object in the input image data, measuring the dis- 
tance from the detected object to a predetermined po- 
sition, and detecting a predetermined object on the ba- 
sis of the measurement result. 
[0016] According to another preferred embodiment, 
there is provided an image processing apparatus/meth- 
od characterized by inputting image data by image pick- 
up means having an optical system, detecting an object 
in the input image data, controlling the optical system of 
the image pickup means, and detecting a predeter- 
mined object on the basis of the object detection result 
and the optical system control result. 
[0017] Other features and advantages of the inven- 
tion will become apparent from the following detailed de- 
scription taken in conjunction with the accompanying 
drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0018] 

Fig. 1 is a view showing an example of the relation- 
ships between the image sizes of an object and the 
distances of the same object from a camera; 
Fig. 2 is a view showing the image sizes of the ob- 
ject in the individual states shown in Fig. 1 ; 
Fig. 3 is a view showing the relationships between 
the image pickup magnifications and the field an- 
gles; 

Fig. 4 is a view showing an image taken at a certain 
fixed field angle by a video camera; 
Fig. 5 is a view showing a frame in which a detection 
area is set in the image shown in Fig. 4; 
Fig. 6 is a view for explaining object detection in the 
detection area set as shown in Fig. 5; 
Fig. 7 is a view for explaining object detection in the 
detection area set as shown in Fig. 5; 
Fig. 8 is a block diagram showing the arrangement 
of a moving object detection apparatus according 
to the first embodiment of the present invention; 
Fig. 9 is a view showing an example of a focus de- 
tection area; 

Fig. 10 is a view for explaining a moving object de- 
tection method using background difference; 
Fig. 11 is a block diagram showing the arrangement 
of a moving object detection unit 5 shown in Fig. 8; 
Fig. 1 2 is a block diagram showing the arrangement 
of a noise removal unit 56 shown in Fig. 11; 
Fig. 1 3 is a view for explaining a 3 x 3-pixel area set 
to remove noise; 

Fig. 1 4 is a view showing noise-removed binary im- 
age data input in raster scan order; 
Figs. 15A and 15B are views for explaining a rec- 
tangular area 81; 



Fig. 1 6 is a block diagram showing the arrangement 
of a moving object size detection unit 6 shown in 
Fig. 8; 

Fig. 1 7 is a block diagram showing a system control 

s unit 20 shown in Fig. 8; 

Fig. 18 is a flow chart for explaining the process of 
detecting a moving object of a predetermined size; 
Fig. 1 9 is a flow chart for explaining the process of 
moving object position detection; 

to Fig. 20 is a flow chart for explaining the process of 
detecting the distance to a moving object; 
Fig. 21 is a flow chart for explaining the process of 
moving object size detection and correction; 
Fig. 22 is aflow chart for explaining another process 

15 of moving object size detection and correction; 

Fig. 23 is a block diagram showing the arrangement 
of a moving object detection apparatus according 
to the fifth embodiment of the present invention; 
Fig. 24 is a block diagram showing the arrangement 

20 of a system control unit 200 shown in Fig. 23; and 
Fig. 25 is a flow chart for explaining the process of 
detecting a moving object within a certain distance 

25 DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0019] Embodiments of the present invention will be 
described below with reference to the accompanying 
so drawings. 

[0020] Fig. 8 is a block diagram showing the arrange- 
ment of a moving object detection apparatus according 
to the first embodiment of the present invention. Refer- 
ring to Fig. 8, a phototaking lens 12 with a zooming func- 
35 tion includes a zooming lens 1 for changing the magni- 
fication and a focusing lens 2 for focusing. This photo- 
taking lens 12 forms an optical image of an object on 
the imaging surface of an image pickup element 3 such 
as a CCD. The image pickup element 3 outputs an elec- 
ta trical signal indicating the optical image to a camera 
processing unit 4. The camera processing unit 4 per- 
forms well-known processes (e.g., gain correction, y 
correction, and color balance adjustment) for the output 
from the image pickup element 3 and outputs a video 
is signal of a predetermined format. A focus detection area 
setting unit 7 designates an image area to be automat- 
ically focused by a focusing control unit 8. 
[0021] Fig. 9 is a view showing an example of the fo- 
cus detection area in an image. In Fig. 9, it is assumed 
50 that an image area 31 is a digital image composed of 
640 x 480 pixels. A rectangular area composed of 140 
x 1 40 pixels in this image area 31 is indicated as a focus 
detection area 32. A circular object 33 is shown as a 
principal object present in the focus detection area 32. 
55 [0022] The focus detection area 32 is relative posi- 
tional information on the imaging surface, which indi- 
cates an area in the imaging surface having an object 
image to be focused. This focus detection area 32 is set 
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in the focusing control unit 8 via the focus detection area corresponds to a previously assumed size is checked, 
setting unit 7 under the control of a system control unit thereby checking whether the moving object is a desired 
20. The focusing control unit 8 moves and adjusts the object having a size to be detected. In this operation a 
position of the focusing lens 2 along its optical axis by moving object size correction unit 21 corrects the size 
controlling a focusing lens motor (stepping motor) (not s 0 f the detected moving object to a size obtained when 
shown) so as to maximize a high-frequency component the object is detected at a reference distance and a ref- 
contained in a portion of the output image signal from erence magnification, in accordance with the distance 
the camera processing unit 4, which corresponds to the from the camera to the moving object. This allows de- 
area set by the focus detection area setting unit 7, there- tection of a moving object of a specific size within a 
by automatically focusing the focusing lens 2 on the ob- 10 broader range than in conventional methods, 
ject. The position of the focusing lens 2 (the lens position [0026] Details of the moving object detection unit 5 
at any arbitrary timing within a position range over which will be described below with reference to Fig. 1 1 . Refer- 
the focusing motor can drive the focusing lens) is exter- ring to Fig. 11 , a video capture unit 51 receives the out- 
nally output from the focusing control unit in the form of, put video signal from the camera processing unit 4 
e.g., a pulse number indicating the number of pulses by is shown in Fig. 8 and writes a digital image in units of 
which the focusing lens motor is driven from its refer- frames in a frame memory 52. A background memory 
ence position. 53 stores an image, such as the background image 41 
[0023] A magnification setting unit 9 sets a target taken with no moving object present, which is previously 
magnification when a zoom control unit 10 moves and picked up by an initializing circuit (not shown) before 
adjusts the position of the zooming lens 1 along its op- 20 monitoring is started. 

tical axis by driving a zooming lens motor (stepping mo- [0027] A difference operation unit 54 receives pixel 
tor) (not shown), thereby controlling zooming. This mag- values obtained by simultaneously reading out corre- 
nification setting unit 9 receives a set magnification from sponding pixels of the two images held in the frame 
the system control unit 20 and sets the set magnification memory 52 and the background memory 53 in the scan- 
in the zoom control unit 10 as a zooming motor driving 2s n j n g order and outputs values (absolute values) ob- 
pulse value corresponding the magnification as a zoom- tained by subtracting the output pixel values from the 
ing lens motor control set value. In accordance with this background memory 53 from the output pixel values 
set value, the zoom control unit 10 controls the zooming from the frame memory 52. When the differences (ab- 
lens motor to move and adjust the position of the zoom- solute values) in one frame output from the difference 
ing lens 1 along its optical axis and thereby enables im- 30 operation unit 54 are arranged in the scanning order, the 
age formation zoomed to a desired magnification. Sim- difference image 43 shown in Fig. 10 is obtained, 
ilar to the focusing lens 2, the position of the zooming [0028] A binarizing unit 55 binarizes the output from 
lens 1 is externally output from the zoom control unit 10 the difference operation unit 54 by using a predeter- 
in the form of, e.g., a pulse number indicating the mined threshold value regarded as a significant value 
number of pulses by which the zooming lens motor is 3S and sequentially outputs pixel values, 1 (ON: black) for 
driven from the reference position. Note that the ele- pixels in an area in which the two images have a signif- 
ments described above are well-known elements in a icant difference and 0 (OFF: white) for other pixels, in 
video camera and the like. the scanning order. From this binary image, a noise re- 
[0024] A moving object detection unit 5 detects a mov- moval unit 56 removes, e.g., isolated pixels, fine black 
ing object in an image from the output video signal from 40 pixel areas, and fine holes (fine white pixel areas in con- 
the camera processing unit 4. As a moving object de- tinuous black pixel areas) produced by noise mixed for 
tection method of this sort, a method using background various causes during the processes described so far. 
difference is known. That is, as shown in Fig. 1 0, an im- [0029] Fig. 1 2 shows the arrangement of the noise re- 
age 41 containing no moving object in an observation moval unit 56. Latches 601 to 609 shown in Fig. 12 hold 
area is previously picked up and stored. Next, a monitor 4S bit data (1 bit x 9 pixels = 9 bits) of nine pixels corre- 
image 42 (an image currently being picked up) obtained sponding to a 3 x 3-pixel area 600 as shown in Fig. 1 3. 
during observation is compared with the image 41 to Each of FIFO memories 61 and 62 holds data corre- 
produce a difference image 43 by calculating the differ- sponding to the number of pixels on one scanning line, 
ence between the pixel values of each pair of corre- That is, the FIFO 61 holds input data one line before the 
sponding pixels. This difference image 43 has significa- so current scanning line. The FIFO 62 holds input data two 
tion pixel values only in a portion different from the im- lines before the current scanning line, 
age 41 previously stored and containing no moving ob- [0030] Of the nine latches 601 to 609, the latches 601 
ject. An area 44 contained in the difference image 43 to 603 hold bit data corresponding to three pixels on the 
and composed of pixels having significant pixel values current scanning line. The latches 604 to 606 hold bit 
(values much larger than zero) is detected as a moving 55 data corresponding to three pixels on a scanning line 
object, adjacent (in the subscanning direction) to the current 
[0025] Whether the size (e.g., the number of pixels scanning line. The latches 607 to 609 hold bit data cor- 
contained in the area 44) of the detected moving object responding to three pixels on a scanning line adjacent 
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(in the subscanning direction) to the scanning line cor- 
responding to the latches 604 to 606. Consequently, da- 
ta is sequentially shifted in units of pixels in synchronism 
with sequential input of the output binary image data 
from the binarizing unit 55 to raster scanning lines. This 
realizes sequential scanning of the image in the area of 
3x3=9 pixels. 

[0031] A ROM 63 receives the nine output bits from 
the latches 601 to 609 as address input and outputs 1 -bit 
data in accordance with the states of the nine output bits 
from the latches 601 to 609. The ROM 63 previously 
holds data by which the ROM 63 outputs 1 when five 
bits or more of the nine input address bits are 1 and out- 
puts 0 when five bits or more of the nine input address 
bits are 0. That is, the ROM 63 is so set as to output 
black pixels when five pixels or more in the 3 x 3-pixel 
area are black pixels and output white pixels when four 
pixels or less in the area are black pixels. Isolated pixels 
can be removed by using the ROM 63 as a lookup table 
as described above. This noise removal unit 56 is a pipe- 
line processing circuit, so an output is delayed by one 
scanning line and by one pixel from the input. However, 
binary pixels from which noise is already removed are 
sequentially output in the raster scan order. 
[0032] Referring to Fig. 11, a bit map memory 57 
stores the binary pixel data of one frame output from the 
noise removal unit 56. A moving object position detec- 
tion unit 58 sequentially receives the noise-removed bi- 
nary image data in the raster scan order as shown in 
Fig. 14 and detects coordinate values (X min ,Y min ) and 
(X max ,Y max ) indicating a rectangular area 81 surround- 
ing a black pixel area as shown in Figs. 15A and 15B. 
These coordinate values can be easily detected by a 
known circuit basically including counters and compa- 

[0033] That is, four counters and four buffers are pre- 
pared to detect and hold X min , X max , Y min , and Y raax . A 
counter for detecting X min counts pixels in the main 
scanning direction until a black pixel appears for the first 
time in data on each scanning line (i.e. , counts synchro- 
nizing pulses (not shown) in the main scanning direc- 
tion). A comparator compares this count with a value 
counted on previous scanning lines and held in a buffer 
for holding X min . If the counter value is smaller than the 
buffer value, the value held in the X min buffer is replaced 
with the current count; if not, the value held in the X min 
buffer is not changed. The value of the X min buffer is 
initialized to a value larger than the number of pixels 
contained in one main scanning line every time a line is 
scanned. 

[0034] To obtain X max , it is only necessary to detect 
pixel position on a main scanning line when a white pixel 
is detected after a black pixel is detected on the scan- 
ning line (i.e., to count main scanning synchronizing 
pulses until a change from a black pixel to a white pixel 
is detected). If this X max value is larger than a previous 
X max value, the X max value is updated; if not, the X^ 
value is not updated. To obtain Y min , it is only necessary 



to count scanning lines (subscanning synchronizing 
pulses) scanned before a scanning line containing a 
black pixel is first detected. To obtain Y max , it is only nec- 
essary to count scanning lines before a scanning line 

s containing no black pixel is again detected after a scan- 
ning line containing a black pixel is detected. 
[0035] When one frame of the binary image is thus 
completely scanned, the coordinates (X min ,Y min ) and 
( x max. Y max) of tne diagonal points of the rectangular ar- 

io ea surrounding the moving object can be detected. 
[0036] A moving object size detection unit 6 shown in 
Fig. 8 detects the size of the moving object on the basis 
of the output values (X min ,Y min ) and (X max ,Y max ) from 
the moving object detection unit 5 and the noise-re- 

is moved binary image data held in the bit map memory 
57 shown in Fig. 11. 

[0037] Fig. 16 shows the arrangement of the moving 
object size detection unit 6. 

[0038] Referring to Fig. 1 6, a scanning clock genera- 
te tion unit 91 receives the coordinates (X min ,Y min ) and 
(X max , Y max ) of the diagonal points of the rectangular ar- 
ea surrounding the moving object from the moving ob- 
ject detection unit 5 and sequentially generates (in a 
raster scan form) addresses for accessing only the rec- 
25 tangular area in a bit map memory 92. 

[0039] That is, the scanning clock generation unit 91 
generates scanning clocks for (X max - X min + 1 ) pixels 
from X min to X max in the main scanning direction and 
scanning clocks for (Y max - Y min + 1 ) scanning lines from 
so Y mjn to Y max in the subscanning direction, thereby con- 
verting the area 81 shown in Fig. 1 5A into a binary image 
82, shown in Fig. 15B, composed of (X max - X min + 1) x 
( Y max • Y min + 1 ) P ixsls - A counter 94 counts only black 
pixels (i.e., when black pixels are output as pixel value 
35 1 and a white pixels are output as pixel value 0, counts 
only pixel values 1 output in this raster scan form) in the 
binary image having (X max - X min + 1 ) x (Y max - Y min + 
1 ) pixels output from the bit map memory 92. In this man- 
ner the counter 94 counts the number of black pixels as 
40 the area of the extracted moving object. This number 
(area) of black pixels is the moving object size. 
[0040] An initialization/read-out unit 93 initializes the 
scanning clock generation unit 91 and the counter 94 
under the control of the system control unit 20 shown in 
45 Fig. 8. Also, the initialization/read-out unit 93 reads out 
the count from the counter 94 and outputs the readout 
count to the system control unit 20. 
[0041] A distance measurement unit 11 shown in Fig. 
8 will be described below. This distance measurement 
so unit 11 receives a focusing lens motor driving pulse 
number (a pulse number indicating the number of pulses 
by which the focusing lens motor is driven from the ref- 
erence position to the current position). This pulse 
number is output from the focusing control unit 8 and 
55 indicates the position of the focusing lens 2. The dis- 
tance measurement unit 11 also receives a zooming 
lens motor driving pulse number (a pulse number indi- 
cating the number of pulses by which the zooming lens 
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motor is driven from the reference position to the current 
position). This pulse number is output from the zoom 
control unit 1 0 and indicates the position of the zooming 
lens 1. The distance measurement unit 11 outputs the 
distance from the camera to an object on which the cam- 
era is focusing. 

[0042] The image pickup lens 1 2 with a zooming func- 
tion shown in Fig. 8, which includes the focusing lens 2 
facing the imaging surface of the image pickup element 
3 and the zooming lens 1 on the object side, is called a 
rear focus lens. For this rear focus lens, the focal point 
moves when the position of the zooming lens 1 is 
changed. Accordingly, an in-focus image can be ob- 
tained only when the focusing lens 2 is also moved. 
[0043] When the rear focus lens is used, therefore, 
the position (i.e., the focusing lens motor pulse number) 
of the focusing lens 2 is changed to various values, and 
the distance from the camera to an object to be focused 
is previously actually measured for each of these posi- 
tions. A lookup table is formed which receives the posi- 
tion (the zooming motor driving pulse number required 
to move from the reference position) of the zooming lens 
1 and the position (the focusing lens motor driving pulse 
number required to move from the reference position) 
of the focusing lens 2 as addresses and outputs the cor- 
responding distance from the camera to an object to be 
focused as data. This lookup table is implemented by a 
ROM. 

[0044] Assuming that both of the zooming motor driv- 
ing pulse number and the focusing motor driving pulse 
number can take on values from 0 to 2,047, the memory 
space is 2K x 2K = 4M (2 11 x 2 11 = 2 22 ), and the data 
dynamic range is 8 bits. That is, when the measurement 
resolution has 256 values and the range of 0 mm to ~ 
is expressed by 256 different distances, the lookup table 
can be formed by a ROM having a capacity of 4 MBytes. 
The data dynamic range can also be 16 bits or the like 
where necessary. If this is the case, the focusing dis- 
tance is expressed by one of 65,536 different distances 
within the range of 0 mm to °°. 
[0045] Fig. 17 shows the arrangement of the system 
control unit 20. 

[0046] Referring to Fig. 17, the system control unit 20 
includes a CPU 22, a ROM 23 storing programs as a 
storage medium according to the present invention, a 
RAM 24, I/O ports 25 to 29, a communication interface 
39, and a bus 30. The CPU 22 reads out the programs 
stored in the ROM 23 and operates in accordance with 
the program procedures. In the course of the operation, 
the CPU 22 holds information required to be temporarily 
held and information changing in accordance with the 
situation in the RAM 24. As the storage medium, it is 
also possible to use a semiconductor memory, an opti- 
cal disk, a magnetooptical disk, or a magnetic medium. 
[0047] The I/O ports 25, 26, 27, 28, and 29 interface 
the CPU 22 with the moving object detection unit 5, the 
moving object size detection unit 6, the focus detection 
area setting unit 7, the distance measurement unit 11, 



and the magnification setting unit 9, respectively. The 
communication interface 39 communicates with exter- 
nal apparatuses. For example, the communication in- 
terface 39 receives the size of a moving object to be 

s detected from an external apparatus or, when a moving 
object with a desired size is detected, informs an exter- 
nal apparatus of the detection. 
[0048] A series of operations of detecting a moving 
object of a known size will be described below with ref- 

10 erence to a flow chart shown in Fig. 18. These opera- 
tions are performed by the CPU 22 by reading out pro- 
gram procedures stored in the ROM 23 and executing 
the programs. 

[0049] When the process is started in Fig. 18, in step 
is S1 the CPU 22 receives a desired magnification D from 
an external host computer via the communication inter- 
face 39. The CPU 22 sets the input magnification D in 
the magnification setting unit 9 via the I/0-5 (29) shown 
in Fig. 17. As described previously, the magnification 
so setting unit 9 causes the zoom control unit 10 to control 
the zooming lens motor in accordance with the magni- 
fication D and sets the desired magnification D in the 
apparatus. 

[0050] After step S1, the flow advances to step S2, 

25 and the CPU 22 receives a size S of a moving object to 
be detected from the external host computer via the 
communication interface 39. The CPU 22 holds the in- 
put size information S in a predetermined area of the 
RAM 24. As this moving object size S, the number of 

30 pixels (in the case of the sphere 124 shown in Fig. 1, 
approximately 84,000 pixels contained in the sphere 
1 24 shown in Fig. 2) at a field angle when the camera 
used in this system are set at a reference distance (in 
this embodiment, 1 m) and a reference magnification (in 

35 this embodiment, a magnification when the horizontal 
field angle is 36° and the vertical field angle is 27° is a 
reference magnification of 1x) is input. 
[0051] The flow then advances to step S3, and the 
CPU 22 starts a loop (steps S3 to S6) of detecting a 

40 moving object with a desired size. Fig. 1 9 shows details 
of step S3. Referring to Fig. 1 9, in step S30 the CPU 22 
accesses the moving object detection unit 5 via the l/O- 
1 (25) and reads out the diagonal point coordinates 
( x mi,v Y min) and (X max ,Y ma)( ) of the rectangular area 81 

45 surrounding a moving object from the moving object po- 
sition detection unit 58. The flow advances to step S31 , 
and the CPU 22 calculates the coordinates 

(Xc, Yc) of the central point of the rectangular area 
81 surrounding the moving object by 

so 

Xc = ( X max- X mi n) /2 



Yc = (Y max - Y ■ )/2 



on the basis of the readout coordinates ( x ra j n .Y min ) and 

OW Y max ). 
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[0052] The flow then advances to step S32, and the 
CPU 22 sets the coordinates (Xc.Yc) of the central point 
of the rectangular area surrounding the moving object, 
which are calculated in step S31 , in the focusing control 
unit 8 via the l/P-3 (27) and the focus detection area 
setting unit 7. In this manner the CPU 22 sets the moving 
object as a focused object of distance measurement by 
the distance measurement unit 1 1 . After completing the 
series of processes in step S3, the CPU 22 returns to 
the routine shown in Fig. 18, and the flow advances to 
step S4. In step S4, the CPU 22 detects the distance to 
the moving object. 

[0053] Fig. 20 shows details of step S4. Referring to 
Fig. 20, in step S40 the CPU 22 receives a signal indi- 
cating whether the focusing control unit 8 determines 
that the moving object is in-focus via the focus detection 
area setting unit 7, thereby checking whether the mov- 
ing object is in-focus. If the moving object is not in-focus, 
the CPU 22 repeats the process in step S40. If the mov- 
ing object is in-focus, the flow advances to step S41 . In 
step S41 , the CPU 22 reads out information Lo about 
the distance to the focused object from the distance 
measurement unit 11 via the I/0-4 (28) shown in Fig. 17. 
The flow then advances to step S42. As described ear- 
lier, Lo expresses distance from 0 mm to °° as an 8- or 
1 6-bit code. I n step S42, therefore, the CPU 22 decodes 
the distance Lo to a distance L (m) from the camera to 
the focused object on the basis of a correspondence ta- 
ble (not shown) previously registered in the program. 
[0054] After completing the series of processes in 
step S4, the CPU 22 returns to the routine shown in Fig. 
18, and the flow advances to step S5. Fig. 21 shows 
details of step S5. 

[0055] Referring to Fig. 21 , in step S50 the CPU 22 
accesses the moving object detection unit 6 via the l/O- 
2 (26) shown in Fig. 17 to receive a moving object size 
(pixel number) So. The flow advances to step S51 , and 
the CPU 22 calculates a size S', which is supposed to 
be obtained when the object is imaged at the reference 
distance of 1 .0 m and the reference magnification, by 

S' = So X (L/D) 2 

on the basis of the magnification D input in step S1 , the 
distance L from the camera to the focused object calcu- 
lated in step S42, and So input in step S50. 
[0056] After completing the series of processes in 
step S5, the CPU 22 returns to the routine shown in Fig. 
18, and the flow advances to step S6. In step S6, the 
CPU 22 checks whether the ratio of the size S' calculat- 
ed in step S51 to the size S of the moving object to be 
detected, which is input in step S2 and held in the RAM 
24, falls within a predetermined range. That is, the CPU 
22 checks whether 



0.8<S7S<1.2 



thereby checking whether the moving object has a de- 
sired size. 

[0057] If the moving object does not have a desired 
size, i.e., if 

0.8>S7S 



1.2<S7S 

the flow returns to step S3, and the CPU 22 repeats the 
is procedure from moving object detection. 

[0058] If the moving object has a desired size, i.e., if 

0.8<S'/S<1.2 

20 

the flow advances to step S7. 
[0059] in step S7, the CPU 22 informs the external 
apparatus of the detection of the moving object with a 
desired size via the communication interface 39. 
25 [0060] In this manner the CPU 22 completes the pro- 
cedure of detecting a moving object of a predetermined 
size. 

[0061 ] Note that the constants 0.8 and 1 .2 used in this 
embodiment can also be adjusted to, e.g. , 0.75 and 0.25 

so or 0.85 and 1.15 in accordance with the components 
used and the use environment. 
[0062] In the above first embodiment, the moving ob- 
ject size input in step S2 of the flow chart shown in Fig. 
18 is not necessarily limited to the number of pixels of 

35 an object as a moving object to be detected, which is 
obtained when the object is apart the reference distance 
from the camera and the field angle of the camera is set 
at the reference magnification. That is, in the second 
embodiment of the present invention, a desired size 

40 composed of a horizontal size Xw (pixels) and a vertical 
size Yw (pixels) of a rectangular area surrounding an 
object as a moving object is input as the moving object 
size while the object is being imaged. 
[0063] In the second embodiment, the parameter to 

45 be corrected in step S5 is not a moving object size So 
obtained from a moving object size detection unit 6 but 
the information relating to a rectangular area surround- 
ing a moving object, which is obtained from a moving 
object detection unit 5. That is, the details of step S5 are 

so changed to a flow chart shown in Fig. 22. 

[0064] Referring to Fig. 22, in step S50a a CPU 22 
calculates 

55 X ° = X max " X min 



Yo = Y max - Y min 
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on the basis of diagonal point coordinates (X^Y,,,,,,) 
and (X raax ,Y max ) of a rectangular area surrounding a 
moving object, which is input from the moving object de- 
tection unit 5 in step S30 described above. In this way 
the CPU 22 calculates a width Xo in the horizontal di- 
rection and a height Yo in the vertical direction of the 
rectangular area. 

[0065] The flow then advances to step S51 a, and the 
CPU 22 calculates a horizontal size Xo' and a vertical 
size Yo' of the rectangle surrounding an object, which is 
supposed to be obtained when imaging is performed at 
a reference distance of 1 .0 m and a reference magnifi- 
cation, by 

Xo' = Xo X (LVD) 



Yo' = Yo X (LVD) 

on the basis of a magnification D input in step S1 , a dis- 
tance L f rom the camera to the focused object calculated 
in step S42, and Xo and Yo calculated in step S50a. 
[0066] After completing the series of processes in 
step S5, the CPU 22 returns to the routine shown in Fig. 
18, and the flow advances to step S6. 
[0067] In step S6 of this embodiment, the CPU 22 
checks whether the ratios of Xo' and Yo' calculated in 
step S51a to Xw and Yw input in step S2, respectively, 
fall within predetermined ranges. That is, the CPU 22 
checks whether 

0.8<Xo'/Xw< 1.2 

and 

0.8 < Yo'/Yw < 1 .2 
thereby checking whether the moving object has a de- 

[0068] If both of Xo' and Yo 1 satisfy the above condi- 
tions, the CPU 22 determines that a moving object with 
a desired size is detected; if not, the CPU 22 determines 
that no such moving object is detected. 
[0069] The constants 0.8 and 1.2 described above 
can also be changed to, e.g., 0.85 and 1 . 1 5 or 0.75 and 
1.25. 

[0070] In this embodiment, the moving object size de- 
tection unit 6 need not count the number of pixels occu- 
pied in an image by a moving object. This simplifies the 
circuit configuration. 

[0071] The moving object size input in step S2 of the 
flow chart shown in Fig. 18 is not necessarily limited to 
the form disclosed in the first embodiment. That is, in 
the third embodiment of the present invention, actual di- 
mensions of an object as a moving object to be detected 



are input as the moving object size. 
[0072] In this third embodiment, a vertical dimension 
(height) H (m) and a horizontal dimension (width) W (m) 
when an object is viewed front ways are input as actual 

5 dimensions. If the field angle is 36.0° and the distance 
from thecamera to the object is 1 m, the horizontal width 
in an image is about 0.65 m, and this width is input by 
using 640 pixels. Accordingly, the relationship between 
the horizontal dimension (width) W (m) and a horizontal 

?0 size Xw, explained in the second embodiment, of a rec- 
tangle surrounding an object at a reference distance and 
a reference field angle (magnification) explained in the 
second embodiment is given by 

75 Xw=(W/0.65) X 640 

[0073] A vertical field angle of 27.0° of the camera cor- 
responds to a width of about 0.48 m in an image taken 
20 at the reference distance, and this width is input by using 
480 pixels. Therefore, a vertical size Yw, described in 
the second embodiment, of the rectangle surrounding 
the object at the reference distance and the reference 
field angle (magnification) can be calculated by 

2S 

Yw= (H/0.48) X 480 

[0074] As described above, actual dimensions are in- 
30 put as the moving object size in step S2 and converted 
into Xw and Yw on the basis of the above equations. 
The rest of the operation is exactly the same as in the 
second embodiment. 

[0075] In this embodiment, the size of a moving object 
3S can be input regardless of the specifications of a camera 
system. This improves the operability of the system. 
[0076] Values in certain ranges can also be input as 
the desired moving object size described in the second 
embodiment. That is, in the fourth embodiment of the 
40 present invention, it is determined that a moving object 
has a desired size if the value of Xw satisfies 

Xw min <Xw<Xw max 

4S 

This similarly applies to Yw. 

[0077] In step S6, it is determined that a moving object 
has a desired size if 

50 0.8 < Yo'/Yw ma)< and Yo'/Yw min < 1 .2 

55 0.8 < Xo7Xw max and Xo7Xw min < 1 .2 

[0078] More specifically, in the first embodiment, 
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0.8 < S'/S and S'/S- < 1 .2 



is checked. 

[0079] In the third embodiment, H min , H max , W min , and 
W max satisfying 



w min < W < w max 

are input to calculate 

Yw max = (H max /0.48) X 480 

Yw min = (H min /0.48) X 480 

Xw max = (W max /0.65) X 640 

Xw min = (W mjn /0.65) X 640 

On the basis of the above equations, deformation as in 

the second embodiment is performed. 

[0080] This embodiment can handle an elastic, easily 

deformable moving object or a moving object which 

changes its size in accordance with the image pickup 

direction. 

[0081] In each of the above embodiments, the size of 
a detected moving object is corrected on the basis of 
the magnification and the distance to the object. How- 
ever, it is also possible to correct previously given infor- 
mation pertaining to the size of a moving object to be 
detected. 

[0082] In each embodiment, a series of processes are 
complete if warning of detection of a moving object with 
a desired size is output as shown in the flow chart of Fig. 
18. However, the present invention is not limited to the 
above embodiments, so moving object detection can al- 
so be repeatedly executed. That is, the flow can also 
return to step S3 even after step S7 in Fig. 18 is com- 

[0083] In the first to fourth embodiments as described 
above, a moving object of a particular size can be de- 
tected in a broader monitoring area than in conventional 
systems by using information indicating the distance to 



the moving object detected from an image, information 
regarding the size of the moving object to be detected, 
and information regarding the size of the moving object 
detected from an image. Also, a moving object of a spe- 

s cific size can be detected even when the magnification 
is varied. Furthermore, the above effects can be 
achieved with a simpler arrangement by using focusing 
control information in distance measurement. 
[0084] The fifth embodiment of the present invention 

10 relates to a moving object detection apparatus for de- 
tecting a moving object in a predetermined distance 
range. 

[0085] Fig. 23 is a block diagram showing the ar- 
rangement of the moving object detection apparatus ac- 

is cording to the fifth embodiment of the present invention. 
The same reference numerals as in Fig. 8 denote parts 
having the same functions in Fig. 23, and a detailed de- 
scription thereof will be omitted. 
[0086] In this embodiment, the process of a system 

so control unit 200 differs from that of the moving object 
detection apparatus shown in Fig. 8. This difference will 
be described below. 

[0087] Fig. 24 shows the arrangement of the system 
control unit 200. 
2S [0088] Referring to Fig. 24, the system control unit 
200 includes a CPU 220, a ROM 230, a RAM 240, and 
a bus 300. The CPU 220 reads out programs stored in 
the ROM 230 and operates in accordance with the pro- 
gram procedures. In the course of operation, the CPU 
30 220 holds information required to be temporarily held 
and information changing in accordance with the situa- 
tion in the RAM 240. An I/O port (1 ) 250 interfaces the 
CPU 220 with a moving object detection unit 5. 
[0089] I/O ports (3, 4, and 5) 270, 280, and 290 inter- 
as face the CPU 220 with a focus detection area setting 
unit 7, a distance measurement unit 11 , and a magnifi- 
cation setting unit 9, respectively. A communication in- 
terface 390 communicates with external apparatuses. 
For example, the communication interface 390 receives 
40 the size of a moving object to be detected from an ex- 
ternal apparatus or, when a moving object with a desired 
size is detected, informs an external apparatus of the 
detection. 

[0090] A series of operations of detecting a moving 
45 object in a predetermined distance range will be de- 
scribed below with reference to a flow chart shown in 
Fig. 25. These operations are performed by the CPU 
220 by reading out program procedures stored in the 
ROM 230 and executing the programs. Note that the 
so ROM 230 can be, e.g., a semiconductor memory, an op- 
tical disk, a magnetooptical disk, or a magnetic medium. 
[0091] When the process is started in Fig. 25, in step 
S101 the CPU 220 receives a desired magnification D 
from an external host computer via the communication 
55 interface 390. The CPU 220 sets the input magnification 
D in the magnification setting unit 9 via the I/0-5 (290) 
shown in Fig. 24. As described previously, the magnifi- 
cation setting unit 9 causes a zoom control unit 10 to 
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control a zooming lens motor in accordance with the 
magnification D and sets the desired magnification D in 
the apparatus. In step S102, the CPU 220 receives a 
distance range Lq-L, (m) to a moving object to be de- 
tected from the external host computer via the commu- s 
nication interface 390. The CPU 220 holds the input dis- 
tance range Lq-L, in a predetermined area of the RAM 
240. L 0 and L, represent the distances from an image 
pickup unit in the optical axis direction (the direction of 
depth) of a lens and satisfy Lg < L, . That is, a moving io 
object to be detected is an object in the distance range 
of L 0 to L, from the image pickup unit. The flow then 
advances to step S103, and the CPU 220 starts a loop 
(steps S103 to 105) of detecting a moving object in the 
predetermined distance range. 75 
[0092] Step S1 03 is the same as in the process pro- 
cedure shown in Fig. 1 9, so a detailed description there- 
of will be omitted. When a series of processes in step 

5 1 03 are complete, the flow advances to step S 1 04, and 
the CPU 220 detects a distance L to a moving object. zo 
[0093] Step S104 is the same as in the process pro- 
cedure shown in Fig. 20, so a detailed description there- 
of will be omitted. When a series of processes in step 

5104 are complete, the flow advances to step S105. 
[0094] In step S1 05, an inside and outside distance 2s 
range discrimination unit 210 in Fig. 23 checks whether 

the moving object is in the predetermined distance 
range by checking whether L 0 < L < L v If L 0 < L < L 1 , 
this means that the moving object is in the predeter- 
mined distance range, so the flow advances to step so 
S106; if not, this means that no moving object is in the 
predetermined distance range, so the flow returns to 
step S103, and the CPU 220 again executes the loop of 
detecting a moving object in the predetermined distance 
range. In step S106, the CPU 220 informs the external 35 
apparatus of the detection of a moving object in the pre- 
determined distance range via the communication inter- 
face 390. 

[0095] In this manner the CPU 220 completes the pro- 
cedure of detecting a moving object in a predetermined 40 
distance range. 

[0096] The predetermined distance range designa- 
tion method in the fifth embodiment is not restricted to 
designation of L 0 and (m). For example, in the sixth 
embodiment of the present invention, a predetermined 4S 
distance Lc (m) and its nearby range AL (m) which are 
related to L 0 and L, in the fifth embodiment as: 



are input. 

[0097] Also, the unit of numerical values need not be 
in meters. That is, it is of course possible to use a value 
expressed by an 8- or 16-bit code which is used when 



the distance from a camera to an object to be focused 
is obtained by using an LUT, as explained earlier as the 
arrangement of the distance measurement unit 11. Fur- 
thermore, distance data need not be input from an ex- 
ternal host computer via the communication interface 
390. For example, the video camera main body can in- 
clude dial switches or ten-key buttons (not shown), and 
an operator can directly designate data by using these 
switches or buttons. 

[0098] An image pickup lens 12 in the fifth embodi- 
ment need not have a zooming function. In the seventh 
embodiment of the present invention, an image pickup 
lens having no zooming function is used. 
[0099] In this embodiment, the magnification setting 
unit 9 and the zoom control unit 1 0 shown in Fig. 23, the 
I/0-5 (290) shown in Fig. 24, and step S101 shown in 
Fig. 25 are unnecessary. The position and focusing dis- 
tance of a focusing lens are actually measured in ad- 
vance to generate data of an LUT in a distance meas- 
urement unit 11 . Addresses of the LUT are input by us- 
ing driving pulses of a focusing lens motor. In this em- 
bodiment, a more inexpensive arrangement than in the 
fifth embodiment is possible, although no variable mag- 
nification can be set. 

[0100] In the fifth to seventh embodiments of the 
present invention as described above, the distance to a 
moving object in an image is measured and compared 
with information pertaining to a predetermined distance 
range. Consequently, a moving object in the predeter- 
mined distance range can be reliably detected. 
[01 01] The above effect can be achieved with a sim- 
pler arrangement by using information regarding focus- 
ing control or zoom control in the distance measure- 
pi 02] In other words, the foregoing description of 
embodiments has been given for illustrative purposes 
only and not to be construed as imposing any limitation 
in every respect. 

[0103] The scope of the invention is, therefore, to be 
determined solely by the following claims and not limited 
by the text of the specifications and alterations made 
within a scope equivalent to the scope of the claims fall 
within the true spirit and scope of the invention. 



An image processing a| 



js comprising: 



a) input means for inputting image data: 

b) object detecting means for detecting an ob- 
ject in the input image data from said input 
means; 

c) measuring means for measuring a distance 
from the object detected by said object detect- 
ing means to a predetermined position; and 

d) predetermined object detecting means for 
g a predetermined object on the basis 
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of an output from said measuring means. 

2. An apparatus according to claim 1 , wherein said 
predetermined object detecting means detects an 
object whose distance to the predetermined posi- 
tion falls within a predetermined range. 

3. An apparatus according to claim 2, wherein said in- 
put means comprises image pickup means for pick- 
ing up an image of an object via an optical system. 

4. An apparatus according to claim 3, wherein the pre- 
determined position is a position of said image pick- 
up means. 

5. An apparatus according to claim 3, wherein 

said image pickup means comprises focusing 
control means for controlling focusing of said 
optical system, and 

wherein said measuring means measures the 
distance from the object detected by said object 
detecting means to the predetermined position 
on the basis of focusing control information 
from said focusing control means. 

6. An apparatus according to claim 1 , further compris- 
ing size detecting means for detecting a size of the 
object detected by said object detecting means, 

wherein said predetermined object detecting 
means detects an object with a predetermined size 
on the basis of an output from said size detecting 



An apparatus according to claim 6, wherein said 
predetermined object detecting means comprises 
setting means for setting a size of an object to be 



distance from the object detected by said object 
detecting means to the predetermined position 
on the basis of focusing control information 
from said focusing control means. 

1 0. An apparatus according to claim 1 , f u rther compris- 
ing output means for outputting a detection output 
from said predetermined object detecting means to 
an external apparatus. 

11. An apparatus according to claim 10, wherein when 
said predetermined object detecting means detects 
a predetermined object, said output means outputs 
the detection result to said ext« 



8. An apparatus according to claim 6, wherein 40 

said input means comprises image pickup 
means for picking up an image of an object via 
an optical system, 

said image pickup means comprising zoom 45 
control means for controlling said optical sys- 
tem to enlarge an image, and 
wherein said predetermined object detecting 
means detects an object with the predeter- 
mined size on the basis of zoom control infor- so 
mation from said zoom control means. 

9. An apparatus according to claim 8, wherein 

said image pickup means comprises focusing 55 
control means for controlling focusing of said 
optical system, and 

wherein said measuring means measures the 



1 2. An apparatus according to claim 1 , wherein said im- 
age processing apparatus is incorporated into a 
monitoring camera. 

13. An apparatus according to claim 3, wherein said 
measuring means uses control information for con- 
trolling said optical system of said image pickup 



14. An apparatus according to claim 3, wherein said 
predetermined object detecting means uses control 
information for controlling said optical system of 
said image pickup means. 

15. An image processing apparatus comprising: 

a) image pickup means having an optical sys- 
tem; 

b) object detecting means for detecting an ob- 
ject in image data picked up by said image pick- 
up means; 

c) control means for controlling said optical sys- 
tem of said image pickup means; and 

d) predetermined object detecting means for 
detecting a predetermined object on the basis 
of an output from said object detecting means 
and an output from said control means. 

i. An apparatus according to claim 1 5, wherein said 
predetermined object detecting means detects an 
object within a predetermined distance range from 
said image pickup means. 

17. An apparatus according to claim 16, wherein 

said control means controls focusing of said op- 
tical system, and 

wherein said predetermined object detecting 
means uses focusing control information from 
said control means. 

18. An apparatus according to claim 15, further com- 
prising size detecting means for detecting a size of 
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the object detected by said object detecting means, 
wherein said predetermined object detecting 
means detects an object with a predetermined size 
on the basis of an output from said size detecting 
means. s 

19. An apparatus according to claim 18, wherein 

said control means controls zooming of said op- 
tical system, and 10 
wherein said predetermined object detecting 
means uses zooming control information from 
said control means. 

20. An apparatus according to claim 15, further com- is 
prising output means for outputting the detection re- 
sult to an external apparatus when said predeter- 
mined object detecting means detects a predeter- 
mined object. 

21. An apparatus according to claim 15, wherein said 
image processing apparatus is incorporated into a 
monitoring camera. 

22. An image processing method comprising the steps 25 



a) inputting image data; 

b) detecting an object in the input image data; 

c) measuring a distance from the detected ob- 30 
ject to a predetermined position; and 

d) detecting a predetermined object on the ba- 
sis of the measurement result. 

23. An image processing method comprising the steps 35 



a) inputting image data from image pickup 
means having an optical system; 

b) detecting an object in the input image data; 

c) controlling said optical system of said image 
pickup means; and 

d) detecting a predetermined object on the ba- 
sis of the detection result in the object detection 
step and the control result in the control step. 
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FIG. 10 
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FIG. 16 
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